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TRENDS  IN  THE  OUTPUT  AND 
IMPACT  OF  RESEARCH  PAPERS 
FROM  THE  UK,  USA,  JAPAN, 
GERMANY  AND  FRANCE 


This  booklet  summarises  the  findings  of  a  study  into  the 
position  of  UK  cardiovascular  research  internationally.  The 
analysis  focused  on  cardiovascular  research  published  in 
refereed  journals  and  covered  tens  of  thousands  of  papers,  in 
an  attempt  to.  provide  a  reliable  broad  brush  picture  of  trends 
in  international  activity  in  the  field. 


S 


Full  details  of  the  study  are  available  in: 

O'Driscoll  M,  Rogers  L  and  Anderson  J  1 994.  Cardiovascular  research  1 98 1  - 1 99 1 :  international  comparisons  of 
research  outputs  in  the  United  Kingdom,  United  States,  Japan,  Germany  and  France.  PRISM  report  No.  8,  the 

Wellcome  Trust,  London. 

Rogers  L  A  and  Anderson  J  1993.  A  new  approach  to  defining  a  multidisciplinary  field  of  research: 
the  case  of  cardiovascular  biology.  Scientometrics  Vol.  28  No.  I  61-77 
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Background  to  the  analyses 


Data  sources 

Science  Citation  Index 
(SCI):  1981-1986 
This  covers  over  3000 
journals,  of  which  1500  are 
biomedical.  It  is  a  key 
database  for  'bibliometric' 
analysis,  because  it  includes 
the  addresses  of  all  authors 
on  each  paper,  so  permitting 
analysis  of  research  collabo- 
ration through  international 
co-authorship.  It  is  also  the 
only  database  to  carry 
citations  to  papers,  so 
permitting  analysis  of 
research  impact.  However,  at 
the  time  of  this  study,  data 
suitable  for  analysis  were 
only  available  up  to  1986. 
The  SCI  was  therefore  only 
used  for  analysing  interna- 
tional collaboration  and 
citation  impact  during  the 
early  1980s. 

Medline:  1981 -199 1 

This  covers  over  3500 
biomedical  journals.  Its 
drawback  is  that  during  the 
1980s  it  did  not  carry  the 
addresses  of  all  authors  on 
each  paper.  Its  strength  is 
that  it  has  a  very  detailed 
indexing  system  (Medical 
Subject  Headings  -  MeSH), 
making  it  possible  to  identify 
sets  of  papers  in  specific 
biomedical  disciplines  like 
cardiovascular  research. 


Defining  the  field 

This  was  an  iterative  process, 
involving  the  following 
stages: 

•  creating  a  'filter'  of  MeSH 
index  terms  through  which 
papers  in  Medline  were 
screened 

•  consulting  with  an  expert 
panel  of  cardiovascular 
researchers  to  check  for 
irrelevant  and  missing  areas 

•  validating  the  final  set  of 
terms  used  to  define  the 
boundary  of  the  field,  with  a 
series  of  empirical  tests  on 
university  publications  data. 

This  process  resulted  in  a 
definition  of  the  cardio- 
vascular field  which  captured 
basic  and  clinical  research 
papers,  drawn  from  a  wide 
range  of  biomedical  disci- 
plines. Importantly,  the 
definition  was  shown  in 
validation  tests  to  exclude 
papers  outside  the  cardio- 
vascular field,  as  well  as 
include  those  which  were 
relevant.  Full  details  of  the 
process  of  field  definition  are 
given  in  the  paper  enclosed 
in  the  back  pocket  of  this 
leaflet  (Rogers,  L  and 
Anderson,  J  Scientometrics, 
28,  61-77,  1993). 


Having  defined  the  field 
through  Medline,  and 
identified  a  set  of  tens  of 
thousands  of  cardiovascular 
papers  in  this  database,  the 
set  was  matched  to  the  SCI 
to  get  fuller  bibliographic 
details  (e.g.  addresses)  on 
each  paper. 


Section  I  !  International  Trends 


Fig  I 


Biomedical  and  cardiovascular  publications  for  all  countries:  1981-1991  (Medline  analysis) 
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Figure  1  depicts  the  growth  in  total 
number  of  biomedical  papers 
worldwide  throughout  the  1980s, 
and  shows  the  proportion  of  these 

Fig  2 


that  were  captured  as  cardiovascular 
research,  using  the  MeSH  definition 
created  in  this  study. 

Overall,  cardiovascular  papers 


accounted  for  1 9%  of  biomedical 
papers,  or  an  average  of  65  000 
papers  per  year,  over  the  period 
studied. 


Biomedical  and  cardiovascular  publications  in  five  countries:  1988-1991  (Medline  analysis) 
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Figure  2  presents  data  for  five 
countries.  It  reveals  substantial 
activity  in  the  US,  which  published 
over  four  times  more  papers  than  the 
UK.  This  reflects  the  huge  output  of 
biomedical  papers  by  the  US 
generally  which  increased  from  25% 
of  all  papers  in  Medline  in  1988  to 


30%  in  1991.  In  contrast,  the  UK 
produced  only  a  steady  6%  of  the 
world's  biomedical  papers  annually 
between  1988-1991. 

In  terms  of  the  share  of  the  world's 
cardiovascular  papers,  the  US  had  an 
average  of  27%,  the  UK  5%, 
Germany  5%  and  France  3%  over 


the  period.  The  only  country  to 
exhibit  a  cleat  increase  in  its  share  of 
the  world's  output  of  cardiovascular 
publications  over  the  period  was 
Japan  (from  6%  to  8%). 


Fig  3  3 
International  comparison  of  cardiovascular  publications  activity  indices:  1988-1991  (Medline  analysis) 
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Figure  3  shows  the  emphasis  placed  1.00  suggests  lower  than  expected 

on  cardiovascular  research  by  different  activity.  From  1988  to  1991  Japan 

countries  in  relation  to  their  overall  and  Germany  consistently  had  activity 

national  activity  in  biomedical  indices  above  1 .00,  suggesting  that  in 

research.  these  countries  cardiovascular  science 

To  show  this  clearly,  it  is  necessary  had  some  prominence  within  their 

to  compute  a  'cardiovascular  research  national  research  portfolios, 

activity  index'  for  each  country  for  In  contrast,  in  the  UK,  US  and 

each  year,  using  the  formula  above.  France  cardiovascular  research  had 

An  index  greater  than  1 .00  reveals  a  less  prominence,  with  low,  and 

greater  than  expected  output  of  declining  activity  indices.  By  1991, 

cardiovascular  research  in  relation  to  indices  for  the  UK  and  US  had  fallen 

the  world  norm,  whilst  a  value  below  below  1.00.  Thus,  while  the  UK  and 

Fig  4 

Citation  impact  of  cardiovascular  papers:  1981-1986  (SCI  analysis) 
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US  publish  large  numbers  of  cardio- 
vascular papers  each  year,  it  is  clear 
that  this  field  does  not  stand  out  in 
these  countries,  as  it  does  in  Japan 
and  Germany. 

Whilst  the  number  of  papers 
produced  by  a  country  gives  some 
measure  of  the  productivity  of  its 
cardiovascular  research  enterprise,  the 
number  of  citations  received  by  these 
papers  gives  some  measure  of  the 
country's  impact  or  influence  in  this 
field  internationally.  Figure  4  shows 
the  results  of  such  an  analysis. 
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Citations  per  paper  for  country  A 

Relative  Citation  Index  =   

(RCI)  Citations  per  paper  for  all  countries 


Here  the  average  number  of  citations 
per  cardiovascular  paper  for  a  country 
is  compared  to  the  world  average  cita- 
tions per  cardiovascular  publication, 
using  a  Relative  Citation  Index.  Given 
that  citation  analysis  could  only  be 


conducted  using  the  SCI,  the  analysis 
is  limited  to  the  period  1981-1986. 
Values  above  1 .00  denote  a  citation 
impact  highet  than  the  world  norm. 

By  this  measure,  US  and  UK 
cardiovascular  research  papers  had 


above  average  impact  internationally. 
In  contrast,  although  Japan  and 
Germany  appeared  in  recent  years  to 
be  relatively  active  in  cardiovascular 
research  (Fig  3),  the  evidence  of 
figure  4  is  that  these  countries  were 
having  below  average  citation  impact 
in  this  field,  at  least  during  the  first 
half  of  the  1980s. 
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International  comparison  of  the  emphasis  on  different  subfields:  1981-1986  (SCI  analysis) 
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In  an  attempt  to  assess  whether 
there  were  any  differences  between 
countries  in  their  research  activity 
within  specific  subfields,  data  were 
disaggregated  according  to  the  main 
Medline  index  terms  making  up  the 
definition  of  cardiovascular  research. 

Each  paper  in  Medline  usually 
carries  several  index  terms,  denoting 
that  its  subject  matter  spans  several 
subfields.  Analysis  of  subfield 
activity  which  includes  such  papers 
will  necessarily  mean  that  they  are 
'counted'  in  two  or  more  of  the 
subfields.  Such  an  analysis  (Fig  5), 
therefore,  can  only  show  the  total 
frequency  with  which  different 
MeSH  terms  were  used  by  Medline 
indexers  to  describe  the  subject 
matter  of  papers  from  a  particular 
country.  However  with  the  large  set 
of  papers,  available  in  this  study,  this 
kind  of  analysis  provides  a  rough 
guide  to  the  relative  activity  in 
different  subfields  in  different 
countries. 


The  data  were  expressed  as 
Subfield  Activity  Indices,  using  the 
formula  above.  An  index  greater 
than  1.00  suggests  a  concentration  of 
effort  in  a  particular  subfield. 

The  most  striking  result,  as  shown 
in  figure  5,  is  that  Japan  appeared  to 
be  relatively  inactive  in  research  on 
cardiovascular  surgery  and  most 
active  in  research  on  cardiovascular 
agents.  Differences  are  less  clear  for 
the  UK,  but  the  subfields  with 
greatest  relative  activity  appear  to 
have  been  physiology  and  cardiovas- 
cular agents,  and  the  least  active 
subfields  to  have  been  diagnostic 
techniques  and  cardiovascular 
surgery.  In  Germany,  the  subfield 
with  the  highest  relative  activity  was 
research  on  diagnostic  techniques, 
and  in  the  US  it  was  research  on 
cardiovascular  surgery. 


Fig  6 


International  collaboration  in  cardiovascular  research:  1981-1986  (SCI  analysis). 
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As  research  is  becoming  increasingly 
internationalised,  it  was  of  interest  to 
assess  patterns  of  collaboration 
between  the  key  countries  in  cardio- 
vascular research.  The  degree  to  which 
scientists  in  one  country  collaborate 
with  those  in  another  can  be 
measured  by  examining  the  addresses 
on  each  paper.  On  the  SCI,  this  can 
be  done  electronically.  Figure  6  shows 
the  strength  of  collaboration  between 
the  UK  and  the  four  other  nations 
studied. 

The  co-authorship  index  reveals 
whether  the  UK's  cardiovascular 
researchers  collaborate  with  another 
country  more  than  would  be 
expected,  given  the  total  number  of 
UK/foreign  collaborations,  and  the 
number  of  papers  produced  by  the 
UK  and  the  country  being  studied. 
The  higher  the  index,  the  stronger  the 
association  between  the  UK  and  that 
country. 


The  intensity  of  UK  collaboration 
with  France  and  Germany  is  higher 
than  expected,  given  the  number  of 
papers  produced  and  co-authored  by 
these  3  countries  internationally.  In 
contrast,  collaboration  between  the 
UK  and  US  was  lower  than  expected 
given  the  volume  of  research  activity 
in  that  country.  This  finding  appears 
to  suggest  that  the  degree  to  which 
the  UK  collaborates  with  another 
country  in  cardiovascular  research 
may  be  influenced  by  its  geographic 
proximity. 


Section  2    UK  research  activity 

Fig  7 


An  attempt  was 
made  to  analyse  the 
distribution  of 
publications  across 
institutions  within 
the  UK.  The  only 
reliable  data  source 
for  this  was  the  SCI, 
because  this  gives  the 
addresses  of  all 
authors  on  all  papers. 

This  first  analysis 
(Fig  7)  was  at  the 
level  of  individual 
institutions  and 
limited  to  the  period 
1981-1986.  This 
shows  the  total 
number  of  cardiovas- 
cular papers 
produced,  and 
citations  received 
through  this  period, 
ranked  according  to 
their  average  number 
of  citations  per 
paper. 


The  25  top  producers  of  publications  relevant  to  cardiovascular  science:  1981-1986  (SCI  analysis) 
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BPMF  =  British  Postgraduate  Medical  Federation.  It  is  a  school  of  the  University  of  London  and  comprises  8  research  and  teaching  institutes,  the  departments  of 
Postgraduate  Medicine  and  Dentistry  for  the  4  Thames  Health  Regions  and  a  central  administrative  office.  Papers  where  only  the  BPMF  was  specified  were 
attributed  to  the  BPMF.  Where  a  specific  institution  falling  under  the  BPMF  umbrella  was  listed  the  paper,  or  fraction  thereof,  was  attributed  to  the  institution 
specified. 

The  vast  majority  of  cardiovascular  publications  listed  above  were  produced  by  researchers  working  in  departments  forming  part  of  the  medical  faculty  or 
school  in  each  university.  The  publications  from  teaching  hospitals  which  were  associated  with  each  medical  faculty  or  school  were  also  attributed  to  the 
relevant  university. 

London  medical  schools  are  listed  separately  from  the  University  of  London  publications. 

UL  UNIV  COLLEGE:  All  papers  from  University  College  London  (including  papers  from  University  College  Medical  School)  and  associated  teaching  hospitals. 
UL  KING'S  COLLEGE:  All  papers  from  King's  College  London  (including  papers  from  King's  College  Medical  School)  and  associated  teaching  hospitals. 


The  maps  below  (Figs  8  and  9) 
show  more  recent  data  than  those 
derived  from  the  SCI  (Fig  7),  and 
were  produced  from  a  special 
sampling  technique  using  Medline. 
During  the  period  1988-1991, 
Medline  indexed  14  064  UK 


cardiovascular  papers.  These  carried 
the  names  of  21  000  different 
authors,  which  were  downloaded 
onto  a  computer  file.  The  100  most 
frequent  names  in  this  set  of 
authors  were  identified  as  the  most 
'visible'  cardiovascular  researchers 


in  the  UK  during  this  period.  Their 
institutional  locations  were  identi- 
fied using  various  medical  and 
research  directories. 


Fig  8 


Hot  spots  of  publication  activity  in  UK  cardiovascular  research:  1988-1991  inclusive  (Medline  analysis) 

(Diameter  of  circles  relate  to  the  number  of  highly  visible  researchers  in  country's  top  100) 

The  most  active  centres  outside 
London  were  Cambridge,  with  7  of 
the  most  visible  researchers,  Glasgow 
with  6  and  Oxford  and  Nottingham 
with  5  each. 
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*  Denotes  that  London  is  not  represented 
on  this  map.  It  is  shown  below  in  Fig  9. 


Fig  9 

London  centres  of  publication  activity  in  cardiovascular  research:  1988-1991  (Medline  analysis) 
(Diameter  of  circles  relate  to  the  number  of  highly  visible  researchers  in  country's  top  100) 
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London  dominated  UK  cardiovascular 
research,  with  69  of  the  nations  most 
visible  researchers  working  thete.  The 
biggest  centre  was  RPMS  with  13 
researchers  in  the  top  100,  then 
NHLI  with  12  and  St  George's  Hosp. 
Med.  Sch.  with  10. 


Key:  CRC  Clinical  Research  Centre  (MRC)  ICH 
Institute  of  Child  Health  IN  Institute  of  Neurology 
NHLI  National  Heart  and  Lung  Institute  RFH  Royal 
Free  Hospital  RPMS  Royal  Postgraduate  Medical 
School  UCL  University  College  London  UMDS 
United  Medical  and  Dental  School  (Guy's  and  St 
Thomas's) 


St  George's 


=  10  highly  visible  researchers 


Section  3    Summary  of  findings 


8  This  analysis  was  based  on  a 

very  large  sample  of  research 
papers  which  effectively 
represented  the  total 
production  of  papers 
worldwide.  At  this  macro 
level,  broad  patterns  of 
research  activity  could  be 
discerned  which  help  set  the 
UK's  cardiovascular  research 
effort  in  a  global  context,  and 
provide  some  insights  into 
the  distribution  of  cardio- 
vascular research  within  the 
UK  not  possible  at  a  lower 
level  of  analysis.  The  key 
findings  are  listed  below: 

•  Using  a  carefully 
constructed  definition  for  the 
field,  1  in  5  of  all  biomedical 
research  papers  published 
worldwide  were  identified  as 
having  relevance  to  cardio- 
vascular research 

•  The  US  dominated  cardio- 
vascular research  in  terms  of 
numbers  of  papers  produced 
worldwide.  Japan  was  the 
second  most  prolific 
producer  of  papers  among 
the  five  countries,  and  over 
recent  years  has  been 
increasing  its  lead  over  the 
UK.  The  UK  produced  more 
cardiovascular  papers  than 
the  two  other  European 
nations;  Germany  and 
France. 

•  Cardiovascular  research 
appeared  to  have  some 
prominence  in  the  national 
research  portfolios  of  Japan 
and  Germany.  In  contrast, 
cardiovascular  research  did 
not  stand  out  as  a  specially 
active  field  within  the  UK's 
national  research  portfolio. 


•  Japan  appears  to  have 
conducted  little  research  on 
cardiovascular  surgery  and 
emphasised  research  with  a 
focus  on  cardiovascular 
agents. 

•  The  UK  and  US  had  the 

highest  citation  impact  inter- 
nationally. Citation  rates  to 
cardiovascular  papers 
produced  in  these  countries 
were  higher  than  the  world 
citation  norm  for  this  field. 
France,  Germany  and  Japan 
were  below  the  norm. 

•  The  UK  had  greater  than 
expected  collaboration  in 
cardiovascular  research  with 
Germany  and  France,  given 
the  level  of  research  activity 
in  these  countries.  This 
suggests  that  geographic 
proximity  may  be  an 
important  factor  in  research 
collaboration. 

•  Within  the  UK,  cardio- 
vascular research  was  concen- 
trated heavily  in  London. 
Between  1988  and  1991, 
sixty  nine  of  the  top  100 
authors  of  cardiovascular 
papers  were  based  in  the 
capital. 
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